DOCUMENT RESUME 



ED 297 078 

AUTHOR 
TITLE 

INSTITUTION 

PUB DATE 
NOTE 
PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



UD 026 310 

Proller, Norman L. 

Evaluation of the 1986-87 ECIA Chapter II Teaching 
Enrichment Activities to Minorities Project. 
Dade County Public Schools, Miami, FL. Office of 
Educational Accountability. 
Nov 87 
15p. 

Reports - Evaluative/Feasibility (142) 
MFOl/PCOl Plus Postage. 

Academic Achievement; *Advanced Placement Programs; 
Elementary Education; *Enrichment; *High Achievement; 
♦Minority Group Children; *Program Effectiveness; 
Program Evaluation; Special Education; Students 
IDENTIFIERS Dade County Public Schools FL 

ABSTRACT 

The 1986-87 Teaching Enrichment Activities to 
Minorities (TEAM) Project provided special education to identified 
minority students who exhibited characteristics that might make them 
eligible for placement into an advanced academic program. The 1986-87 
TEAM Project operated in 16 elementary schools and served 
approximately 550 pupils. An evaluation of the program revealed the 
following: (1) 12 of the 16 TEAM schools used the appropriate 
criteria for admitting students to the project, while four of the 
schools failed to do this; (2) all TEAM teachers developed 
appropriate lesson plans and instructional activities; (3) project 
students, taken as a group, achieved gains in their reading 
comprehension, mathematics computation, mathematics application 
Stanford Achievement Tests scores, and Peabody Picture Vocabulary 
Test scores above and beyond those expected from spending a typical 
year in school; and (4) of the 56 TEAM pupils referred for admission 
to an advanced placement program, 28 had been tested and 21 had been 
accepted. Data are presented on six tables. A list of unofficial 
recommendations is appended. (BJV) 
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Executive Summary 



The 1986-87 Teaching Enrichment Activities to Minorities (TEAM) Project was 
Iinn^nnn"^^"" ^^^P^^^ > t^is its third year of operation, for 

$100,000. Its primary goal concerned the provision of "special instruction to 
identified minority students who. exhibit characteristics that might make them 
eligible for placement into an advanced academic program." In brief, the 
Project attempted to enhance its participants' well -developed intellectual 
skills and remediate those which have been ic'entified as less well-developed. 
By strengthening its pupils' strong cognitive abilities and ameliorating 
weaker ones, the Project hoped to "improve the students' thinking skills and 
thus make them more successful in school situations regardless of their 
placement in gifted, academic excellence, or regular programs." 

The 1986-87 year TEAM Project operated in 16 elementary schools and served 
approximately 550 pupi'/s. 

This evaluation was based on data that were routinely collected by DCPS or 
Project personnel as well as upon information that was gathered strictly for 
this appraisal. Methods/sources employed included the examining of student 
records and teacher lesson plans, the testing of Project participants, the 
observing of the delivery of the TEAM-related curriculum; and the interviewing 
oi TEAM teachers. 

The evaluation addressed the following questions: 

1. To what ex cent did the TEAM youngsters meet the selection criteria out- 
lined in the proposal? 

2. To what extent did the TEAM teachers develop lesson plans and instruc- 
tional activities aimed toward enhancing their respective pupils' 
well -developed thinking ?kills and remediating those which were less- 
developed? 

3. To what extent did the TEAM participants obtain gains on Stanford 
Achievement Test (SAT) scores in reading comprehension and mathematics 
computation above and beyond those expected from normal maturation? 

4. To what extent did the Project children achieve gains on the Peabody 
Picture Vocabulary Test (PPVT) above and beyond those expected from nor- 
mal maturation? 

5. To what extent did TEAM students eventually obtain admittance to a pro- 
gram for the academically talented? 

Results of this evaluation showed that 12 of the 16 TEAM schools used the ap- 
propriate criteria for admitting students to the Project while four of the 
schools failed to do this. Furthermore, the appraisal indicated that all TEAM 
teachers developed appropriate lesson plans and instructional activities, and 
that Project students, taken as group, achieved gains in their reading compre- 
hension, mathematics computation, mathematics application SAT scores and on 
their PPVT scores abo/e and beyond those expected from spending a typical year 
in school. Finally, the study showed that of the 56 TEAM pupils referred for 
admission to an advanced academic program, 28 had been tested and 21 had been 
accepted. 



Description of the Project 



Background ; Numerous children from diverse ethnic backgrounds and/or low-in- 
come families who may possess high academic potential are presently not quali- 
fying for advanced academic programs due to their low scores on the 
standardized tests Dade County Public Schools (DCPS) traditionally uses to 
screen pupils for admittance to such programs. The Florida Department of Edu- 
cation's Incident rate for the gifted is two percent of the total school popu- 
lation. Given this rate, DCPS should have approximately 700 black and about 
850 Hispanic elementary students placed in some type of advanced academic pro- 
gram. DCPS records indicate that the number of minority students presently 
enrolled In such programs is considerably below the predicted figures. 

During the 1984-85 school year, the Teaching Enrichment Activities to Minori- 
ties (T.AM) Project was awarded $38,416 under the Education Consolidation and 
Improvement Act (ECIA), Chapter II to address this problem. The Project was 
refunded for the 1985-86 school year in the amount of $20,000. During the 
first two years of its operation, a substantial proportion of TEAM'S monies 
was used to buy materials for the teachers of TEAM students and to pay for 
substitute-release time so that TEAM teachers could attend TEAM-sponsored 
workshops w-thout their respective schools having to underwrite these costs. 

TEAM'S purpose during those two years was to provide inservice training for 
TEAM teachers and supply them with a spec.'al curriculum and related materials. 
This assistance permitted the teachers to deliver a specialized critical 
thinking skills curriculum to approximately 100 selected minority students ex- 
hibiting characteristics that would make them eligible to be considered for 
placement in an advanced academic program. Its goal was to place as many of 
its participants as possible into such programs. 

Results from the last two evaluations of this Project indicated that its fea- 
tures were implemented as described and as scheduled. Also, these appraisals 
revealed that although its participants showed little academic gain in reading 
and mathematics achievement as measured by their Stanford Achievement Test 
(SAT) scores, the Project did place (during the past two years) 28 percent of 
its students into programs for the academically and/or artistically talented. 

During the 1986-87 school year, TEAM was awarded $100,000 under ECIA, Chapter 
II for the purpose of operational izing the Project in 16 elementary schools 
which had not previously been involved with TEAM. All of the monies for this 
year's Project were to be used to underwrite the salary and benefits of a Pro- 
ject Manager, and to pay for materials and substitute release time for the new 
TEAM teachers. The goals for the 1986-87 TEAM Project as well as the activi- 
ties employed to achieve these goals remained the same as in previous years. 

Description of the Evaluation 

This evaluation was based on "already available" TEAM Project or DCPS data as 
well as upon information that was obtained solely for this study. 
Methods/sources employed included the examining of student records and teacher 
lesson plans, the testing of Project participants, the observing of TEAM 
related classroom materials and the interviewing of TEAM teachers. 

The evaluation addressed the following questions: 

1. To what extent did the TEAM youngsters meet the selection criteria out- 
lined in the proposal? 

1 



2. To what extent did the TEAM teachers develop lesson plans and i/istruc- 
tional activities aimed toward enhancing their respective students' 
well -developed thinking skills and remediating those vhich were 
less-developed? 

3. To what extend did the TEAM participants obtain gains on Stanford 
Achievement Test (SAT) scores in reading comprehension and mathematics 
computation above and beyond those expected from normal school 
maturation? 

4. To what extent did the Project children achieve gains on the Peabo dy 
Picture Vocabulary Test (PPVT) above and beyond those expected from nor- 
mal school maturation? 

5. To what extent did TEAM students eventually obtain admittance to a 
program for the academically talented? 

Results 

Extent to Which the TEAM Youngsters Met the Selection Criteria Outlined in the 
TEAM Proposal 

The TEAM proposal stipulated that "multiple criteria" including but not 
limited to "teacher recommendations, and a teacher checklist which 
characterizes a student's motivation, learning, and creativity" would be 
utilized to select TEAM participants. To determined the extent to which the 
TEAM participants met the criteria for admission to the TEAM Project, 
interviews were conducted with all of the TEAM teachers and an examination was 
made of teacher checklists and all participants' SAT scores. Analyses of 
information collected from these sources showed that two schools used only the 
TEAM checklist, eight schools used both the checkl ist "and student stanine 
scores (Note - in these particular cases stanines were utilized to eliminate 
Chapter I-eligible pupils), four schools used only stanines, one school (that 
did not receive its checklist due to its delayed admittance into the Project) 
used only teacher recommendations; and one school used the checklist, student 
stanine scores, and ethnicity as selection criteria. It should be added, that 
an examination of all TEAM students' stanines in reading comprehension and 
mathematics computation indicated that none of the Project participants were 
eligible for any type of compensatory education program. 

In brief, P of the TEAM schools used either (or both) of the stipulated 
criteria to admit students into the Project, while four schools did not use 
either the checklist or teacher recommendations. 

Extent to Which the TEAM Teachers Developed Appropriate Lesson Plans and In - 
structional Activities to Enhance Students' Wei 1 -Developed Thin king Skills and 
Remediate those Less Wei 1 -Developed 

To determine the extent to which the TEAM teachers developed appropriate les- 
son plans and instructional activities to enhance their students' 
well -developed thinking skills; and remediate those less well -developed, 
inspections were made of all Project teacher's lessons plans and observations 
of teacher activities were made in a 50 percent random sample of TEAM 
classrooms. Analysis of this qualitatively collected data suggested that 
teachers were, indeed, preparing appropriate lessons plans (based upon con- 
cepts they had learned during inservice training), correctly employing the 
TEAM materials (e.g. the Building Thinking 3kill workbook), and offering their 
respective students relevant classroom activities. 
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Extent to Which Project Students Obtained SAT Scores Above Those Expected from 
a Normal School Year Maturation 



To determine the extent to which the TEAM pupils obtained SAT scores above and 
beyond those expected from normal maturation, several norm-referenced analyses 
were performed on the participants' 1986-87 SAT scores in reading comprehen- 
sion, mathematics computation and mathematics application. 

More specifically, based on the participants' 1985-86 performance on the SAT, 
predicted SAT scores for 1986-87 were derived and the difference between the 
obtained and the predicted scores were statistically analyzed. As Table la 
shows, results of these analysts suggested that, taken as a group, the pupils 
obtained statistically significant gains (beyond what would have been 
expected) beyond the .0001 level in reading comprehension, mathematics compu- 
tation, and mathematics application. When the data were broken out by school, 
analyses showed that in 15 of the 16 TEAM schools, the pupils achieved gains 
in reading comprehension that were significantly better than expected. In 
mathematics computation, the TEAM pupils obtained gains that were sign- 
ificantly better than expected in 14 of the 16 TEAM schools. In addition, 
with regard to mathematics application, the students at the four TEAM schools 
where the test was administered also evidenced statistically significant gains 
above and beyond those expected. 

Finally, it should be noted that, as Tables lb and Ic show, when the data were 
broken out by ethnicity or by sex all five subgroupings (e.g., white non- 
Hispanics, black non-Hispanics, Hispanics; and males, and females) obtained 
significant gains from pre to posttest administrations. 

In brief, it seems indisputable that the TEAM Project helped its participants 
achieve SAT scores above and beyond what they would typically have obtained 
from spending a year in school. 

Extent to Which the Participants Achieved Peabod.y Picture Vocabulary Test 
Scores beyond Those Expected from Normal Maturation 

To determine the extent to which the TEAM pupils obtained PPVT scores above 
and beyond those expected from spending a typical year in school, the PPVT was 
administered on a pre and posttest basis to all students involved in the TEAM 
Project. As Table Ila shows, statistical analysis suggested that, taken as a 
roup, the participants achieved significant gai-is beyond the .0001 level 
t=.3423, p< 0.0001). 

Furthermore, as can be observed in Tables lib and lie, when the data were 
broken out by ethnicity or by sex, all five subgroupings (i.e. white 
non-Hispanics, black non-Hispanics, and Hispanics; and males, and females), 
analyses indicated that all achieved statistically significant gains above and 
beyond those expected from spending a typical year in school. 

In short, it appears that the TEAM Project assuredly helped its participants 
obtain statistically significant PPVT scores above and beyond those expected 
from normal maturation. 

Extent to Which Participants Achieved Admittance to Some Type of Advanced Aca- 
demic Program 

To ascertain the extent to which the TEAM .'itudents achieved admittance to some 
type of advanced academic program, an examination was made of Project records 
and an inter lew was conducted with the Director of Advanced Academic Pro- 
^ grams. 
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Table la. 

Project Impact on Participants' SAT Scores 



Subtest 


Post - Pre 

Mean 
Difference 


Standard 
Error of 
the Mean 


t 


Probability 
of 
t 


Reading Comprehension 40,9581 




1 Q A"} 


U.UUUl 


Mathematics Computation 33.9491 


2.9456 


11.53 


0.0001 


Mathematics Application 39.5657 


3.8070 


10.39 


n nnn i 

U.UUUl 




Table lb 










Project's Impact on Participants' SAT Scores 






By Ethnicity 






Ethnicity 


Post-Pre 
Mean 

Subtest Difference 


Standard Error 
of the Mean 


t 


Hrobabi nty 
of 
t 




Reading 

Comprehension 4L543 


5.3049 


8.58 


0.0001 


White non- 
Hispanic 


Mathematics 

Computation 17.6286 


8.9773 


1.96 


0.0578 




Mathematics 

Application 49.5000 


10.7404 


4.61 


0.0025 




Reading 

Comprehension 42.6383 


3.1164 


13.68 


0.0001 


Black non- 
Hispanic 


Mathematics 

Computation 37.3032 


3.9318 


9.49 


0.0001 




Mathematics 

Application 40.6571 


4.7832 


8.50 


0.0001 




Reading 

Comprehens ion 36 . 8349 


3.7956 


9.70 


0.0001 



Hispanic Mathematics 

Comprehension 33.0640 5.1587 6.41 0.0001 



Mathematics 

Application 32.4375 8.1558 3.98 0.0012 
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Table Ic. 

Project's Impact on Participants' SAT Scores 



By Sex 



Gender 


Subtest 


Post - Pre 

Mean 
Difference 


Standard Error 
of the Mean 


t 


Probability 
of 
t 




Reading 
Comprehension 


37.6029 


3.3546 


11.21 


0.0001 


Male 


Mathematics 
Computation 


36.7941 


4.9693 


7.40 


0.0001 




Mathematics 
Application 


35.1538 


5.8425 


6.02 


0.0001 




Reading 
Comprehension 


43.2626 


3.8031 


14.65 


0.0001 


Female 


Mathematics 
Computation 


31.9949 


2.8207 


8.85 


0.0001 




Mathematics 
Application 


42.9412 


3.7169 


8.56 


0.0001 
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Table Ila, 

Project's Impact on Pai-ticipants' PPVT Scores 



Probability 

Mean Standard of 
Difference Deviation Sign Rank t 



1.1061 3,3743 5.9639 O.OOC: 



Table lib. 

Project's Impact on Participants' PPVT Scores 



By Ethnicity 



Ethnicity 


Mean 
Difference 


Standard 
Deviation 


Sign 
Rank 


Probability 
of 
t 


white non- 










Hispanic 


0.9728 


1.1925 


217.5 


0.0001 


black non- 










Hispanic 


1.0135 


4.2051 


7797,5 


0,0001 


Hispanic 


1.3317 


1,8342 


230,0 


0.0001 



Table lie. 

Project's Impact on Participants' PPVT Scores 

By Sex 



Probability 



Mean Standard Sign of 

Se?; Di fference Deviation Rank t 

M^ile 0.6807 4.8196 3942 0.0342 

•■" 1.2653 1.7430 8013 0.0001 
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Information gathered from these sources indicated that 56 of the 447 TEAM 
pupil J were referred for testing to determine if they would qualify for admit- 
tance to an advanced academic program. 

Of the 56 who had been referred for testing, 28 had been tested by the end of 
of May, 1987. Of these 28, 21 had been accepted, and one had been rejected 
froiM the DCPS. gifted program. In essence, about 75 percent of the TEAM pupils 
referred for testing into the gifted program had oeen accepted as of the 
preparation date of this report. This figure compares most favorably with the 
county-wide acceptance rate. 



Conclusions 



Results of this study indicated that the vast majority of schools selected 
their TEAM students according to the proposal guidelines. In addition, the 
evaluation showed that the TEAM teachers developed appropriate lesson plans 
and instructional activities aimed toward enhancing their respective students' 
well-developtd thinking skills and remediating those which were less well-de- 
veloped. Furthermore, the appraisal showed that cpe TEAM Project 
significantly impacted its participants' SAT scores in reading comprehension, 
matiamatics computation, and mathematics application above and beyond that 
expected from spending a typical year in school. Concomitantly, the 
evaluation also showed that the TEAM Project significantly influenced its 
participants' PPVT scores above and beyond what was expected from spending a 
typical year in school. Finally, the study revealed that a substantial per- 
centage of TEAM pupils who were tested for the gifted programs were, indeed, 
accepted. 



BUREAU RESPONSE/RECOMMENDATIONS/PLANS OF ACTJ-^N 
TO ADDRESS EVALUATION FINDINGS* 



* I he following section was developed by program staff and is not an offi- 
cial part of the evaluation report itself. It consists of a bureau/office 
response and a list of actions/recommendations which are to be (or have al- 
ready been) initiated by the relevant bureau/office. The recomnendations 
were generated by program staff using input from the OEA evaluator(s^ who 
conducted the evaluation. ^ ' 
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December 15, 1987 



H85P0H SB TO TOOOMIOil OP nB 1986-67 BdA CHAPXBl H 

iHffliHS Emcatan mzivixus to iii m j hitus (tebm) hcobct 



A review of the 1986-87 BOA Chapter II Teaching Enrictaent Activities to 
T^Z^^^.^T^^ 5^ ^ ccqpleted by the Bureau of Education staf? A^ 

""^^ information contained in the referenced 

report, the following reccmnendations are being m^e: 

1. Continue the project in its present form for the remainder of the 1987-88 
school year. 

2. This program has been funded by Chapter II as a pilot for the past four 
years. It has been axtranely successful. Oonsideration should be given 
S^^S? ? P?^^ ^ ^ systanwide categorical program. Appropriate 
^fftl fS^^^ recamEndations are being developed for consideration 
in the 1987-88 budget process. 

3. The program is based on alternative teaching methods which protote 
critical thinking. This is a great need in the entire school syston but 
especially m the low socio-econonic feeder patterns. A plan will be 
developed to ejqand the program to all low socio-econonic elementary 
schools over the next two to three fiscal years. 

4. The concepts which were successfully tested in this project and the use of 
appropriate materials shcuM be utilized in the junior high/middle schools 
^ 1^ .^u"T. "^^^^^ Specifically, the project should be expanded to 
inclv^e the 16junior high/middle schools which feed the five high schools 
of Miam Northwestern, Miami Central, Miami Jackson, Miami fifison, and 
Miami Carol City. Appropriate budget enhancement recoirendations are 
being developed for consideration in the 1987-88 budget adoption process. 

These recommendations have been reviewed by Dr. Maria de Armas, Mr. Gary Rito, 
of f i«. ^' ^'^^^ 0- White, and have the concurrence of this 



Frank de Varona 
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